Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add projection layer to roberta encoder #1451

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

parmeet
Copy link
Contributor

@parmeet parmeet commented Nov 24, 2021

User may want to additionally project the features from encoder. This PR add support for projecting features to different dimensional space.

@codecov
Copy link

codecov bot commented Nov 24, 2021

Codecov Report

Merging #1451 (157da8b) into main (aea6ad6) will decrease coverage by 0.37%.
The diff coverage is 42.10%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1451      +/-   ##
==========================================
- Coverage   86.38%   86.01%   -0.38%     
==========================================
  Files          58       58              
  Lines        2226     2245      +19     
==========================================
+ Hits         1923     1931       +8     
- Misses        303      314      +11     
Impacted Files Coverage Δ
torchtext/models/roberta/modules.py 78.08% <23.07%> (-4.34%) ⬇️
torchtext/models/roberta/model.py 96.29% <83.33%> (-1.04%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aea6ad6...157da8b. Read the comment docs.

@Nayef211
Copy link
Contributor

Overall LGTM. It seems like we're getting a lot of code coverage warnings. Should we also add unit tests for the ProjectionLayer class?

@parmeet
Copy link
Contributor Author

parmeet commented Nov 25, 2021

Overall LGTM. It seems like we're getting a lot of code coverage warnings. Should we also add unit tests for the ProjectionLayer class?

Yes, I think it's a good idea. The models tests didn't really touch this part of the code since we are not projecting the encoder output for standard architectures. Though, it would be more like integration/smoke tests instead of unit-test. We also do not have unit tests for other modules as well, but they are covered well through our end-2-end model tests (we call them unit-tests for now, but as @mthrok commented in here, we are going to move them to integration tests as well).

@Nayef211
Copy link
Contributor

Overall LGTM. It seems like we're getting a lot of code coverage warnings. Should we also add unit tests for the ProjectionLayer class?

Yes, I think it's a good idea. The models tests didn't really touch this part of the code since we are not projecting the encoder output for standard architectures. Though, it would be more like integration/smoke tests instead of unit-test. We also do not have unit tests for other modules as well, but they are covered well through our end-2-end model tests (we call them unit-tests for now, but as @mthrok commented in here, we are going to move them to integration tests as well).

Gotcha this makes sense. We can have some followup tasks to add these integrations tests. Stamped!

@joecummings
Copy link
Contributor

@Nayef211 @parmeet Is this still relevant?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants